Skip to content

fix: improve catalog and data loading locally#69

Merged
hansmohrmann merged 2 commits intomainfrom
fix-improve-local-file-loading
Nov 21, 2025
Merged

fix: improve catalog and data loading locally#69
hansmohrmann merged 2 commits intomainfrom
fix-improve-local-file-loading

Conversation

@hansmohrmann
Copy link
Collaborator

A couple small tweaks for working with local data:

  1. by moving the DEFAULT_MIRROR out of the arg default we don't explicitly have to set it to None when loading a custom catalog.
  2. For loading parquets locally, by pulling the pd.read_parquet out of the loop, we preserve the hive-partition columns in the path names.

@hansmohrmann hansmohrmann merged commit 8673663 into main Nov 21, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant